1. Field of the Invention
The present invention relates to image reproduction apparatuses and, more particularly, to image reproduction apparatuses that (i) extract arbitrary video/audio data from plural MPEG transport streams that are stored in an apparatus for storing and reproducing digital broadcast data which are composed of MPEG transport streams, and (ii) reorganize a new MPEG transport stream.
2. Description of the Related Art
In recent years, as BS digital broadcasting, terrestrial digital broadcasting, and the like have become widespread, equipment for storing digital broadcast data has become widely available. The digital broadcast data is transmitted in a MPEG transport stream that is defined by ISO/IEC 13818-1. FIG. 17 illustrates a structure of the MPEG transport stream.
The MPEG transport stream is composed of plural packets, each comprising 188 bytes. The 188-byte MPEG transport stream packet is divided into a TS_header and a Payload area.
A packet identifier (PID) in FIG. 17 has a length of 13 bits and is included in TS_header. The PID indicates the type of the packet. The type of the packet (video, audio, or PSI) is decided with reference to PID. In the payload area of a packet having a video PID, video streams are stored.
In addition, adaptation_field_control indicates a structure of data that follow continuity_counter.
In the case where 2 bits of adaptation_field_control are 2′b11, continuity_counter is followed by adaptation_field_length, adaptation_field (when adaptation_field_length=0, there is no adaptation_field), and Payload. In the Payload area, PSI (program specific information) or PES (Packetized Elementary Stream) data are stored.
In the case where 2 bits of adaptation_field_control are 2′b10, continuity_counter is followed by adaptation_field_length and adaptation_field. There is no Payload.
In the case where 2 bits of adaptation_field_counter are 2′b01, continuity_counter is followed by Payload.
The structure of adaptation_field is shown in FIG. 18.
The adaptation_field comprise adaptation_field_length (8-bit data) that indicates the length of adaptation_field, which is followed by flags of 8 bits indicating types of data included in the adaptation_field, which is further followed by Optional_field_data.
Among these flags, the 4th bit from the highest order flag is PCR_flag. When PCR_flag is 1, PCR (Program_clock_reference) is included in Optional_field_data. The PCR indicates the time when the last bit in a packet of the MPEG transport Stream including this PCR would reach a MPEG decoding apparatus. The PCR is constituted by Program_clock_reference_extension (9 bits) and Program_clock_reference_base. Program_clock_reference_extension indicates a value of a counter that counts from 0 to 299 at 27 MHz. When the counter value exceeds 299, the counter returns to 0. Program_clock_reference_base is incremented by 1 at a time when Program_clock_reference_extension exceeds 299. The PCR is used to generate the frequency of a reference clock (herein after, referred to as STC) for the apparatus that decodes the MPEG transport stream.
Further, when OPCR_flag in FIG. 18 is 1, optional_field_area includes the value of OPCR (original_program_clock_reference). OPCR is an area for duplicating the original PCR in generating another MPEG transport stream using a MPEG transport stream.
The structure of the PES data is shown in FIG. 19.
The PES data contains Packet_start_code_prefix starting from 24′h000001, which is followed by Stream_id, flag signals of 2 byte indicating the contents of data included in Optional_field_data, PES_header_data_length indicating the length of PES_header, Optional_field_data, and PES_packet_data_byte.
When PTS_DTS_flags in FIG. 19 are 2′b11, a PTS (presentation time stamp) and a DTS (decoding time stamp) are set in Optional_field_data. The PTS indicates the time when decoded videos or audio are reproduced and outputted. When STC coincides with PTS, images or sounds including this PTS are outputted. The DTS is time information for controlling decoding when the decoding time and the reproduction time are different, like in the case of I pictures or P pictures of MPEG video. The PTS and DTS each have the length of 3 bits, and are indicated in unit of 90 KHz.
When PTS_DTS_flags in FIG. 19 are 2′b10, only the PTS (presentation time stamp) is set in Optional_field_data.
In digital broadcasting, by utilizing the above-mentioned MPEG transport stream format, multiplication of plural images, sounds, and program information such as EPG can be realized. In order to identify the images, sounds and the like, the PID information is employed, and combination of images and sounds is given in the PSI information (PAT, PMT).
Further, in recent years, as the digital broadcasting that is implemented by plural video/audio channels has come into widespread use, systems for simultaneously decoding plural video/audio data have become available. An apparatus for simultaneously decoding plural video/audio data is shown in FIG. 20.
When a MPEG transport stream 6a is inputted to a PID filter 61, the PID filter 61 generates a signal 6b by extracting video/audio packets having a PID to be reproduced, and transmits the generated signal to a buffer 62 and a PCR PTS·DTS detection unit 65. The PCR PTS·DTS detection unit 65 detects PCR in the packet and generates STC. The PCR PTS·DTS detection unit 65 further detects PTS and DTS from the video/audio data. When the STC value coincides with PTS and DTS, the unit 65 transmits a PTS·DTS control signal 6e for controlling a video/audio decoding unit 63 for decoding video/audio data, while when the STC value coincides with PTS, the unit 65 transmits a PTS control signal 6f for controlling displaying of the decoded images or outputting of the decoded sounds. The video/audio decoding unit 63 reads a video/audio packet 6c from the buffer 62 with referring to the PTS·DTS control signal 6e to decode the packet 6c, and transmits decoded video/audio data 6d to a buffer 64. The video/audio data stored in the buffer 64 are transmitted as output images or sounds 6g with reference to the PTS control signal 6f, so that the images and the sounds are synchronized. A decoding apparatus shown in FIG. 20 is constituted by a plurality of the above-mentioned decoding unit.
An example of a screen image that is reproduced by the decoding apparatus of FIG. 20 is shown in FIG. 21.
A display screen image 71 is composed of plural decoded images 72 that are images generated by the decoding apparatus shown in FIG. 20, arbitrary graphics 73, and a program image 74 that shows program information. The program image 74 may be constituted by EPG (Electric Program Guide). It is possible to select whether one of sounds that are decoded by the decoding apparatus of FIG. 20 is selected, an arbitrary sound is reproduced from outside, or no sound is reproduced. By using the decoding apparatus as shown in FIG. 20, plural images can be displayed on one screen. Particularly, because of increases in the capacity of a hard disk drive or an optical disc, it has become possible to simultaneously view and listen to plural MPEG transport streams that were recorded at different times.
In order to obtain a reproduced image as shown in FIG. 21, it is necessary that arbitrary video/audio data should be extracted from among plural MPEG transport streams to form another MPEG transport stream.
In extracting arbitrary video/audio data from among plural MPEG transport streams to form another MPEG transport stream, as time information indicated by PCRs of the respective MPEG transport streams is different, when packets are re-multiplexed as they are, the time information may be changed indiscriminately, whereby it is impossible to decode these packets by the decoding apparatus.
Thus, to solve this problem, the conventional image reproduction apparatus has plural means in parallel for generating a STC from PCR information included in the respective MPEG transport streams, thereby rewriting a PCR of a stream that has been re-multiplexed on the basis of the STC counter (for example, see Japanese Patent Application No. Hei. 11-41193 (FIG. 1) and Japanese Published Patent Application No. 2002-185901 (FIG. 1)).
The conventional method is shown in FIG. 22. FIG. 22 illustrates an apparatus that extracts arbitrary video/audio data from plural MPEG transport streams 2a that are inputted, and outputs a re-multiplexed MPEG transport stream 2g. 
The apparatus shown in FIG. 22 includes plural one-channel reproduction units 25.
In the one-channel reproduction unit 25, MPEG transport streams 2a are inputted to a PID filter 21. The PID filter 21 selects video or audio data of an arbitrary channel from the MPEG transport streams 2a and transmits the selected video or audio data 2b to a buffer 22, as well as selects a packet having a PCR and transmits the selected packet 2c including the PCR to a PCR detection STC generation unit 24. The PCR detection STC generation unit 24 detects the PCR value from the packet 2c including the PCR, and generates a STC so as to be in phases with PCR. A multiplexing means 26 selects video and audio data to be outputted, and tries to read MPEG transport packets that are stored in the buffer 22. At that time, a PCR rewriting unit 23 reads a selected video/audio packet 2d from the buffer 22, further reads a rewriting PCR value 2e that has been read from the PCR detection STC generation unit 24, and rewrites the PCR value in the selected packet to the rewriting PCR value 2e, thereby outputting a reproduction video/audio packet 2f to the multiplexing means 26. Here, in many cases, a STC value that is generated by the PCR detection STC generation unit 24 is employed as the rewriting PCR value 2e. Plural reproduction video/audio packets 2f that are outputted from the plural one-channel reproduction units 25 are multiplexed by the multiplexing means 26, and are outputted as a multiplexed MPEG transport stream 2g. 
However, in the above-mentioned method, two problems would arise.
Initially, since different time base PCR are allocated to respective images that are reproduced by the conventional reproduction apparatus, respective images and sounds must be reproduced by the decoding apparatus using different STC, respectively. Further, when these images are simultaneously displayed on the same screen as shown in FIG. 21, the display becomes complicated because they have different STC.
Secondly, as the PCR is changed at output timing, a difference between PTS·DTS in the stream and the rewritten PCR becomes large, thereby preventing accurate decoding of the MPEG transport stream. Particularly, as the buffer 62 of FIG. 20 controls the remaining amount of the buffer using STC PTS·DTS, when the changed value of the PCR is greatly different from the original PCR, the buffer may overflow or underflow. The buffer overflow or underflow will occur more remarkably as the number of MPEG transport streams to be multiplexed is increased.
As described above, in the apparatus that extracts arbitrary video, audio, and data from among plural MPEG transport streams to be re-multiplexed and reproduced, it is necessary that the PCR should be individually changed for each video or audio to decode the video or audio data at different time bases in the decoding, thereby enabling to display images and sounds on one screen. Further, when the number of MPEG transport streams to be multiplexed is increased, the changed PCR would not coincide with PTS·DTS, resulting in overflow or underflow of the buffer in the decoding apparatus.